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Abstract: As part of a national e-learning benchmarking initiative of the UK Higher Education Academy, the University 
of Manchester is carrying out a pilot study of a method to benchmark e-learning in an institution. The pilot was designed 
to evaluate the operational viability of a method based on the e-Learning Maturity Model developed at the University of 
Wellington, New Zealand, which, in turn was derived from Carnegie Mellon’s widely accepted Capability Maturity Model. 
The method is based on gathering evidence about the many and interdependent processes in the e-learning and student 
lifecycles and takes a holistic view of maturity, addressing multiple aspects. This paper deals with the rationale for the 
selected method and explains the adoption of a process based approach. It describes the iterative refinement of the 
questionnaire used to elicit evidence for measures of five aspects of maturity in a range of e-learning processes, in five 
process areas. The pilot study will produce a map of evidence of e-learning practice across the processes matrix and a 
measure of the degree of embedding in a sample of faculties within the institution expressed as capability and maturity. 
To provide a useful measure of where an organisation is with respect to a particular aspect of e-learning, it needs to be 
able to act on that measure, finding any new activities required or modifying current activities to improve its processes. 
The pilot study aims to evaluate the potential for improvement inherent in the capability maturity model and to examine 
the resource implications of obtaining useful evidence. A successful benchmarking effort should be able to inform an 
institution’s planning and resourcing processes and the outcomes of this pilot should lead to an informed decision about 
a method for benchmarking the embedding of e-learning, both for the particular institution and for the sector, which in 
turn can lead to operational suggestions for improvement. 
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1. Introduction 

A recurring issue in e-learning is the barriers to its 
successful embedding within an institution. What 
is more problematic is the persistence of this 
issue in spite of better project management and 
greater knowledge of the aspects of e-learning. 
Suwardy, Ratnatunga, Sohal, and Speight (2003) 
report similar findings more generally for 
embedding IT, suggesting that the experience is 
not unique to the educational sector. Remeyni, 
Sherwood-Smith, and White (1997) argue that to 
realise business benefits, technology must be 
carefully integrated with the business strategy of 
the enterprise and its corporate culture, a process 
known as Active Benefit Realisation. They 
describe how this involves continuous 
participative evaluation to manage inevitable 
changes in the business, and a shifting of the 
focus of information systems to be the shared 
responsibility of a group of the main information 
systems stakeholders. However, Alshawi, Irani 
and Baldwin (2003) record that managers across 
different sectors continue to struggle to come to 
terms with the socio-technical (human and 
organisational) aspects of IT deployment. They 
note that the benefits from expenditure on IT 
systems continue to be considerably less than 
expected, and that managers continue to find it 


difficult to determine how to evaluate investments 
and realise maximum benefits in IT. 

The same problem exists in the UK Higher 
Education sector. Here most e-learning initiatives 
exist as projects with a finite life. This means that 
single projects or developments can operate only 
as enablers, each of which offers an individual 
focus that can be harnessed to create greater 
organisational change. Success in an organisation 
is however integrally linked to the structure and 
organisation of its environment. It cannot be 
entirely the objective of a limited duration project. 
To be successful a project must have impact on 
the organisation, on the people within it and on 
the way they work. This requires detailed 
knowledge of the organisation and engagement 
with key stakeholders as well as an understanding 
of how project outcomes can be brought in to an 
existing structure. It also requires an organisation 
that is capable of being sufficiently responsive and 
able to accommodate successfully designed 
developments into its structure within normal 
resource constraints. For change to be 
engineered successfully from projects therefore 
requires an organisation-wide approach that 
encompasses all aspects of e-learning. Methods 
for describing and investigating e-learning 
processes within an organisation are not well 
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developed. There is little development of systems 
and processes that are managed across product 
or service lifecycles. In UK HE, teaching and 
learning design, development and delivery are 
generally the product of piecemeal, unconnected 
processes with multiple operators and 
stakeholders, and no consistent quality assurance 
mechanisms. There is need therefore for 
developing the means of describing and analysing 
processes in the e-learning domain that meet the 
requirement of what we might call next generation 
e-learning. That is, e-learning that has moved 
away from cottage industry types of operation to 
those that are scalable, quality assured and 
responsive to market demand. One solution is 
benchmarking processes (Hirumi, 2005) and in 
particular those that provide a guide to change 
and improvement. 

2. Application of a capability maturity 
model to e-learning processes 

The e-Learning Maturity Model (eMM) model was 
developed in New Zealand based on two 
complementary models, the Capability Maturity 
Model (CMM) from the Software Engineering 
Institute (SEI 2002) and SPICE (Software Process 
Improvement and Capability Determination) (El 
Emam, Drouin, and Melo, 1998). The Capability 
Maturity Model for Software characterises a 
mature, capable software process and the 
progression from an immature, ad hoc software 
process to a mature, well-managed software 
process. This model is currently applied to a 
number of industry sectors (Griffiths 2005). 
SPICE, which is a joint effort by the International 
Standards Organisation (ISO) and International 
Electrotechnical Commission (IEC) to create an 
international standard for software process 
assessment adds the approach for organising the 
e-learning provision practices and processes into 
process areas. The CMM has five levels of 
maturity, ranging from ‘initial’ to ‘optimised’. Each 
level of maturity in the CMM has a corresponding 
set of key practices. The practice descriptions are 
an elaboration of what is meant by maturity at 
each level of the CMM. From the first phase of his 
work in New Zealand, Marshall has come to a 
more holistic view of process maturity in which 
there are five dimensions of maturity. There is not 
necessarily a linear progression of capability from 
one to the next. That is, it is not necessary to 
reach full capability in one dimension before 
progressing to the next. It is possible for 
organisations to develop different patterns of 
capability across the five dimensions that are to 
some extent independent (Marshall 2006b). 

The combination of CMM with SPICE as a basis 
for eMM provides a means for an institution to 
appraise their ability to perform their key business 


processes, such as those required for e-learning 
provision. It also provides the mechanism for 
giving guidance to improve process capability. 
The eMM also offers the means to create the 
underlying reference model for measuring process 
maturity from multiple aspects and assessing 
capability within each aspect. Implementing the 
CMM determines the state of an organisation's 
current software process, the high-priority 
software process-related issues facing an 
organisation, and obtains the organisational 
support for software process improvement. 
Implementing the eMM should similarly create a 
picture of the current e-learning provision 
processes across the institution and highlight 
issues facing the HEI. 

In order to succeed with the implementation of 
eMM it is important first to reach an understanding 
of the terms ‘process’ and ‘practice’. A process is 
usually taken to mean the ‘who’, ‘what’, ‘how’ and 
‘when’ of doing something in a context (Kruchten, 
2004). A process in eMM is a description of the 
goal of a set of activities, each of which will be the 
responsibility of some role(s) and carry constraints 
concerning the timing and manner of their 
execution. The activities carried out to achieve the 
five maturity dimensions of that process goal are 
captured in descriptions of practices. Figure 1 
shows how a process has a goal, comprises 
activities, has maturity dimensions that are 
supported by a set of practices and belongs to a 
process area. It may be seen that the practices 
are organised according to which dimension of a 
process they support. eMM seeks to identify the 
dimension and capability of processes from 
evidence about practices. These measures, for a 
set of processes, form the basis for assessing 
institutional (or other level) capability and maturity. 
The processes of interest are those concerned 
with the management of the lifecycles of e- 
learning products, the student experience, and the 
management of the teaching and learning context. 
Figure 1 is a conceptual model of the elements of 
eMM and their connection to the e-learning 
lifecycle which allows a whole system view of e- 
learning provision. We need to understand how 
the eMM processes and practices fit together with 
the e-learning lifecycle and with a process driven 
knowledgebase to support people in their roles 
(Dexter and Petch 2006). Also in this whole 
system view is a monitoring and evaluation 
framework that is required for useful-time 
response to measurements of process and 
product quality which in turn is required to enable 
HEIs to act on the measurement results and make 
the eMM an ongoing process improvement 
mechanism. Future work will be on creating a 
model of the mapping of capability assessment 
into a monitoring and evaluation reference model 
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for the institution’s key processes in teaching and learning and their support. 



Figure 1 Conceptual Model of the eMM and its links to the e- Learning Lifecycle 


3. Characteristics of the eMM method 

The eMM method is a process of assigning values 
to the capability and maturity attributes of the 
process array based on documented and oral 
evidence. eMM considers the whole life cycle of e- 
learning from planning to delivery and evaluation. 
In each of the five process areas that make up the 
life cycle between six and twelve processes are 
identified as key indicators of the process area. 
This process set is, of course, not universally 
fixed. Its selection is a matter of judgement and 
experience and it is subject to revision. For each 
process, a set of practices is identified at each 
level of maturity. Much of the work on applying 
eMM has been on refining and adding to this list 
of practices both from the literature and from local 
instances of practices. In applying eMM, evidence 
on the practices undertaken in each process area 
is collected from the variety of sources that exist 


within an institution. These include quality 
assurance procedures, quality assurance reports, 
course logs, policy and strategy documents, 
course documentation, and a host of other 
documents as well as evidence gained from 
interviewing staff and students. Marshall (2006) 
indicates the breadth and variety of sources that 
are gathered in a typical university. These 
evidence sources are then analysed to determine 
the practices that they provide evidence for so 
that a matrix of evidence against practices can be 
constructed. From this, judgements are made of 
the practices carried out in each of the process 
areas and the levels of maturity of the indicator 
processes. The resultant assessments of maturity 
are then presented in the same form as figures 2 
and 5. The relations between these elements of 
the method are shown in Figure 3. 
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Figure 2 Snapshot of eMM scoring across multiple institutions (from Marshall 2005b) 


The relations between the various elements of the 
method are illustrated in Figure 3 in which it may 
be seen that the set of processes and their 
dimensions form a stable core or standard, and 


the associated practices which are initially drawn 
from the literature may vary according to local 
issues. The evidence collection and mapping for 
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process capability in a particular dimension is 
informed by the practices. 



Figure 3 Core eMM components and their 
relationship to local issues that can modify the 
practices 

The method is intrinsically general. General 
aspects are evident based on first principles and 
on fact that practice matrices come from 
established body of knowledge from the field of e- 
learning. The initial development of eMM by 
Marshall and Mitchell (2004) was based on a 
survey of the literature relating to e-learning, 
quality assurance and benchmarking. Each 
aspect of the metrics, processes and criteria is 
based on an extensive and critical appraisal of the 
literature and as such the method represents the 
best available and widely agreed criteria for 
benchmarking. In choosing eMM, it was 
recognised that the details of the processes, 
practices and evidence needed to be determined 
for each major new instance of use, so that they 
are appropriate to the circumstances of the 
application. In first developing eMM Marshall and 
Mitchell (2004) culled an extensive set of 
practices from the literature. Subsequently, these 
were refined and extended as the method was 
applied to successive cases. A major part of 
developing the application at Manchester was a 
detailed analysis of practices and questions in the 
first version of eMM to produce a version that was 
valid for the university. 

The University of Manchester provides an 
instance of a large research intensive university, 
one of the Russell Group. It has four faculties, 
twenty three schools, over 100 discipline areas 
and 3500 academic and related staff. It has 
recently undergone major reorganisation with a 
merger in 2004. It has high ambitions to be world 
leading site not only in research but also in 
education, based especially on e-learning. It is 
clear that this requires a radical approach to 
managing quality. A stated objective of the 
institution is to establish quality assurance 
processes that are embedded in all aspects of 
education and are not ‘event driven’ as at present. 


Quality is seen as a continuous part of operations 
both to apply and to monitor. Additionally Quality 
Assurance and Enhancement is envisaged as an 
integral part of a planning-review-evaluation cycle 
that is the basis for resource allocation and for 
both operational and strategic planning. This 
requires a method and a toolset that gives not 
only detailed operational intelligence on aspects 
of operations and performance but also provides a 
means of achieving planned and managed 
improvements in aspects of performance. It has in 
other words to be the means to change practice. 
eMM is not a method intended for a one-off 
snapshot of the state of things, but to drive 
change to find where improvements may be made 
and to track change and improvement over time. 

It is a method to allow a visualisation of the state 
of the organisation and compare parts of the 
organisation, or to view state change over time. 
This is an essential aspect for engaging 
operational managers and strategic managers as 
well as academics. Additionally, the value of the 
method as developed by Marshall et al. is in its 
ability to handle and display large amounts of data 
in single graphics (Tufte, 1990). The development 
of rich visual design by Marshall et al. is a 
valuable development of eMM that makes it an 
effective management and communication tool. 
Figure 2 shows how within one display, and this 
only a partial one, around 150 data items are 
received synoptically. The density of data prompts 
a focus on patterns which are the appropriate 
level for institutional analysis and comparison. 
The eMM rationale focuses on the progressive 
changes to achieve optimal performance. At the 
core of the method as it is applied is the set of 
practices that ensure such progression and 
improvement. These are context specific and 
have to be designed and developed for each 
application but there are generic elements relating 
both to HEIs and to e-learning. In particular the 
method cannot be seen simply as applying only to 
e-learning but in reality to all teaching and 
learning. Metrics within the method are designed 
to identify whether or not particular practices that 
may be associated with any level of development 
of processes are followed or not and the extent to 
which they are followed. The practices observed 
in the method are those identified for a particular 
institution, or type of institution, that are ‘indicator’ 
practices. Those, in other words, that give the 
most sensitive indication of differences and 
change. The process of developing the metrics of 
the method is one of fine tuning the indicator 
metrics so as best to reflect the real processes 
involved in design, development and delivery of e- 
learning. This fine tuning is based on 
understanding of processes, on the availability of 
evidence and on the likelihood that evidence is 
reliable. Additionally, the metrics and the criteria 
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used with them are designed to be progressive, 
that is to be sensitive to and indicate changes that 
are real in the ways indicated by the rational of the 
approach, viz. in terms of increasing maturity and 
capability. 

4. Method development: The 
Manchester case 

eMM in its early versions has been developed and 
tested in a limited context outside the UK only on 
project-type operations, and within a limited time- 
frame. In adopting the method at Manchester, 
there was a need to extend eMM to the whole 
institutional level with coverage for the whole e- 
learning lifecycle and to tailor the method to a UK 
HEI. In assessing how eMM can be practically 
applied within a UK HEI, the key components to 
be considered were: 

■ The process list 

■ The database of related practices 

■ The definition of dimensions created through 
assigning practices to the dimensions 

■ The evidence map that defines where 
information on what is being carried out in 
relation to practices can be found 

Marshall’s approach presumes a particular set of 
evidence that should be sought for each 
organisation, and uses questionnaires as a tool to 
locate that evidence. Once a preliminary 
assessment of the available evidence is made, 
interviews can be used to clarify actual 
circumstances against what might be suggested 
from documentation collected up to that point. In 
UK HEIs, the form of evidence and where it may 
be located is less clear. Other local and specific 
factors also contribute to the need for a revised 
approach to guide evidence collection for the pilot 
to be successful in the Manchester case: 

■ A need to assess the scalability potential for 
the eMM methodology, 

■ The ‘newness’ of the organisation under pilot 
and its related structures 

■ The combined impact of old and new 
structures working together in the early stages 
of the organisation 

■ Staff familiarity and day-to-day knowledge in a 
period of extensive change 

■ A requirement to minimise the workload in 
making the methodology repeatable as a 
precursor to long-term viability. 

In collecting evidence, prior contact with 
organisational staff is needed to build an evidence 
matrix to help locate where particular material 
and/or documentation may lie and in what forms it 
may be found. For this purpose, the pilot has 
identified a series of process-based questions 
capable of guiding interview-based discussion. In 
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contrast to Marshall’s approach, interviews are a 
first-contact point for locating evidence, rather 
than a follow up to evidence already identified. 
Questions are used to guide discussion around 
evidence for the processes in eMM. The interview 
process also allows a communication channel to 
be developed with key staff. However, for this 
approach to work, the question set must elicit the 
full set of possible information against all the 
processes listed in the five process areas. 
Otherwise gaps will exist due to the data 
gathering process itself. To achieve such a 
question list requires a multi-dimensional matrix 
mapping of the proposed questions back to the 
eMM process list. It may be that some processes 
and their dimensions may be over-represented, in 
which case the question set can be adjusted. 
Some process dimensions may not be 
represented, which will require questions covering 
these areas to be added. Alternatively, it may be 
found that gaps in questions are in fact covered 
by the evidence. Where questions exist that 
represent a process not already listed in eMM, 
that process may be added for future eMM 
iterations. This set of activities is non-trivial. An 
equivalent exercise is also necessary for deriving 
evidence-based practices appropriate to the pilot 
situation and from which assessments of a 
process can be made. 

The pilot approach developed at Manchester 
should yield a self-contained, comprehensive set 
of questions for producing a structured evidence 
matrix for a UK HEI. What is described here is the 
basis of a transferable technique for adapting 
eMM to take account of differences in local 
structuring and availability of evidence when a 
whole institution is considered. This can be done 
without compromising the integrity or 
comparability of any results generated by this 
method. Initially eMM may not appear to cover 
some issues of immediate concern to UK HE. 
However, there are a number of ways that specific 
educational issues can be investigated within the 
framework: 

■ A process can be identified for inclusion in the 
master process list. 

■ The place of specific issues in individual 
processes can be assessed through the 
practices assigned to each process dimension. 

■ The question list can be modified to include 
specific questions related to the issue. 

Each of these has its own implications for 
continued long-term comparability and integrity of 
eMM and to address this there is a need for a 
continuing dialogue between eMM practitioners. 

To preserve the integrity of eMM as a method, the 
master process list should be subject only to 
minimal changes. In general, practices that reflect 
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specific issues gradually become absorbed. 
Additionally, it is common that practices that at 
one time are regarded as diagnostic of particular 
dimension or levels of maturity may become 
absorbed and then regarded as the norm across 
the sector. They therefore lose their discriminating 
and diagnostic power. Additionally, it is possible 
that as a novel practice becomes part of a routine 
it in fact ceases to indicate any particular level of 
maturity. Given the fluidity of the practice model, it 
is a good approach for specific issues to be 
appropriately aligned and represented within the 
practice database, and continually reviewed. This 
also supports the role of benchmarking as a key 
tool for driving change in practice within an 
organisation. However it is important that the 


integrity of the mapping of a question set to 
processes array is not compromised by random 
addition and removal of questions, which can 
result in gaps and incomplete analysis. Therefore 
it is preferable if changes to the question set are 
done in a version-controlled manner that is 
accompanied by a full re-mapping exercise. 
Figure 4 summarises the pilot project process. It 
identifies each activity (ellipse) in the process of 
piloting the method from initial collection of 
process information and associated practices 
through to the final refined method based on 
evidence from Manchester. The figure shows also 
the artefacts (rectangles) used at each stage as 
well as those produced by the pilot project 
activities. 



Figure 4 The eMM pilot method 
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The principle documented outputs from the pilot 

are: 

■ A set of processes organised into five process 
areas 

■ A set of process related questions used to 
elicit evidence 

■ For each process, a set of practices that are 
used to describe and distinguish level of 
capability at each of its dimensions 

■ For the set of processes, a corresponding 
matrix of evidence that is available for 


determining the nature and status of practices 
across the dimensions. 

Figure 5 illustrates the nature of the processes 
within one of the process areas, that of Learning. 
Ten processes are used to cover this process 
area. Figure 5 also shows an anonymised set of 
results of capability levels for an institution. A 
quick visual inspection reveals those processes 
which are relatively strong or weak and those 
processes where unusual patterns of capability 
are evident. 


Learning 

Processes that directly impact on pedagogical aspects 
of e-learning 

1 

2 

3 

4 

5 

LI. 

Learning objectives are apparent in the design and 
implementation of courses 






L2. 

Students are provided with mechanisms for interaction with 
teaching staff and other students 






L3. 

Student skill development for learning is provided 






L4. 

Information provided on the type and timeliness of staff 
responses to communications students can expect 






L5. 

Students receive feedback on their performance within courses 






L6. 

Research and information literacy skills development by students 
is explicitly supported 

1 





L7. 

Learning designs and activities result in active engagement by 
students 






L8. 

Assessment of students is designed to progressively build their 
competences 






L9. 

Student work is subject to specified timetables and deadlines 






L10. 

Courses are designed to support diverse learning styles and 
learner capabilities 







Figure 5 Patterns of capability across the processes 

An example of the sorts of practices that are used 
to determine such capability levels is given in 
Figure 6 which shows practices relating to the 
Learning process area. 

An example of the evidence that is used to reach 
a judgement about a practice that relates to two 
processes in the Learning Process Area is shown 
in Figure 7. 

5. Reflections on the plot 

The pilot had two related aims. First was to 
provide a vehicle for change management within 
the institution so that it can move to a continuous 
process of improvement in teaching and learning. 


There are already indications of the opportunities 
and challenges in doing this. It is clear that 
institutional buy-in to the change is needed across 
the institution from executive level to course 
development and administration level. Not only do 
practices relate to each level but the whole 
philosophy of the approach assumes a move to a 
connected organisation. The cultural, 
interpersonal, procedural, educational and 
technical aspects of teaching and learning are all 
the subject of change and for the method to be 
effective these aspects of an organisation have to 
be managed. Adopting eMM is not a soft or quick 
option, he second aim was to inform the HE 
sector as part of the sector-wide initiative on 
benchmarking e-learning. The key point here is to 
get over the idea the eMM is one method that has 
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a particular purpose and function. That purpose 
serves well the ambitions of a large university 
seeking radical change and seeking to align 
operational aspects of processes in a systematic 
manner. It will not answer all needs and applied 


inappropriately will be an expensive mistake. For 
those institutions for which it is appropriate the 
pilot will provide a robust, thorough and rational 
method. 


Dimension 

Practice examples 

5: Optimisation 

Information on success of courses as measured by the stated learning outcomes is used to inform and 
support the design and (re)development of courses, programmes and degrees. 

Strategic planning of teaching and learning across the institution is used to determine new or modified 
outcomes that are promulgated to courses, programmes and degrees. 

4: Management 

Information is collected on the extent to which courses are providing learning objectives that address 
the full range of cognitive outcomes appropriate to the course and students. 

Courses are regularly reviewed to ensure that staff are incorporating learning objectives in course 
design and delivery consistent with the expectations of the institutional polcies, guidelines and 
standards. 

Performance of students against the expected outcomes measured using a variety of qualitative and 
quantitative metrics. 

Regular reviews of course learning objectives undertaken to ensure currency and effectiveness. 

3: Definition 

Institutional policies require that a formal statement of learning objectives is part of all course 
documentation provided to students. 

Teaching staff are provided with training, guidelines and examples for developing learning objectives 
that address the full range of cognitive outcomes appropriate to the discipline, pedagogical approach 
and students. 

Teaching staff are provided with training, guidelines and examples in assessing student outcomes and 
the extent to which learning objectives are being met. 

Training, templates, examples, standards and guidelines are provided on howto use learning objectives 
explicitly in the design and delivery of courses in order to assist student learning. 

Insitutionally defined graduate attributes exist and are referenced in policy guiding course, programme 
and degree design, development and delivery. 

2: Planning 

Programme or degree planning and review processes consider the relationship of learning objectives 
of individual courses to those of the programme or degree as a whole 

Statements of learning objectives are explicitly requested in institutional templates for course 
summaries and documents such as course prospectuses or syllabi. 

Course planning and review documentation explicitly refers to the learning objectives when assessing 
the course and making any decisions about the course structure, learning design and content. 

Course design activities reference the learning objectives and use them to determine the nature and 
relationship of content, activities and assessment used in the delivery. 

1: Delivery 

Learning objectives are provided explicitly in the formal descriptions of the course provided to 
students, including the summary versions provided prior to enrolment as well as within detailed course 
prospectuses or syllabi. 

Learning objectives are linked explicitly throughout learning and assessment activities using consistent 
language. 

Learning objectives for individual courses or modules are explicitly linked to wider programme or 
degree objectives and institutional graduate attributes. 

Learning objectives are aimed at supporting student outcomes that go beyond recall and acquisition 
of knowledge. 

Course workload expectations and assessment tasks are consistent with the learning objectives. 


Figure 6 Practices in the Learning Process Area (Marshall 2006a) 
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Learning 

1 

2 

3 

4 

5 


Delivery 

Planning 

Definition 

Management 

Optimised 

LI. Learning 
Objectives aie 
appaient in the 
design and 
implementation 
of the couise 

Course outline 
(C); Courses 
assessment 
description 

(C); 

Course outline 
(C); Courses 
assessment 
description 
(C); Project 
Proposal (PD1 

Faculty e- 
learning 
strategy (S); 
teaching and 
learning 
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Student 
questionnaires 
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and learning 
strategy (S) 


L2. Students aie 
piovided witli 
mechanisms for 
interaction with 
teaching staff 
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students 

Course outline 
(C); Courses 
assessment 
description 
(C); Student 
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E); WebCT 
discussion 
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Student 
support 
website © 

Course outline 
(C); Student 
support 
website (C); 
Student guide 
to e-learning 
© 

Faculty e- 
learning 
strategy (S); 
teaching and 
learning 
strategy (S); 
Netiquette/ 
email policy 
(P); Student 
Feedback (E) 

Student 

questionnaires 

(E); 
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C 

Course content and resources 



S 

Strategies 



P 

Policies 



PP 

Procedures and planning 



R 

Reviews and Reports 



E 

Evaluation & Feedback 



Q 

Quality Assurance 



SD 

Staff Development 



PD 

Project Documents 




Figure 7 A sample of evidence in the Learning Process Area 


One of the merits of the method is the intrinsic life 
cycle approach. No doubt the method could be 
developed to address only limited aspects of the 
lifecycle and especially for the purpose of 
comparison with other benchmarking methods it 
could be applied either partially or redeveloped to 
focus on particular areas with additional metrics 
and criteria. However, this would in some ways 
abandon the logic of the method which is firmly 
rooted within a lifecycle concept. One of the 
characteristics of e-learning that has emerged has 


been a realisation that a lifecycle approach is 
necessary if we are to manage the complex array 
of issues and processes required for successful 
delivery (Dexter, Petch and Wilcox, 2004). The 
eMM method aligns well with this philosophy for 
educational engineering and in fact is seen as one 
part of the repertoire of tools that must underpin 
such an approach. A mapping of eMM on to an e- 
learning lifecycle model based on the Rational 
Unified Process model (Dexter et al. 2004) shows 
that all the main elements of the lifecycle are in 
fact covered by eMM. 
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